Controlled authentication of physical objects

ABSTRACT

A physical object is scanned and a digital image of the object is generated from the scan. At least one subset of the image, known as an “authentication region” is selected. A plurality of feature vectors, arising from the physical structure of the object, are extracted from certain locations of interest within an authentication region and combined to generate a unique identifier or “digital fingerprint” for that object. Preferably, authentication regions for feature vector extraction are automatically selected by a software process. To identify or authenticate an object, a system may compare a digital fingerprint of an object to digital fingerprints previously stored in a database. Digital fingerprint data may specify a set of features (also termed “locations of interest”) which may be referenced in the creation of a “fingerprint template” which is a template of certain locations of interest and/or attributes selected for authenticating a particular class of objects.

PRIORITY CLAIM

This application is a non-provisional of, and claims priority pursuantto 35 U.S.C. §119(e) (2012) to, U.S. provisional application No.62/342,147 filed May 26, 2016, hereby incorporated by reference asthough fully set forth.

COPYRIGHT NOTICE

COPYRIGHT ©2016-2017 Alitheon, Inc. A portion of the disclosure of thispatent document contains material which is subject to copyrightprotection. The copyright owner has no objection to the facsimilereproduction by anyone of the patent document or the patent disclosure,as it appears in the Patent and Trademark Office patent file or records,but otherwise reserves all copyright rights whatsoever. 37 C.F.R.§1.71(d) (2017).

TECHNICAL FIELD

The present disclosure relates to the field of centralized databasesstoring digital fingerprints of objects that enable enhanced security,rapid searching, and high reliability. Other aspects include methods andapparatus to identify, track, and authenticate physical objectsutilizing a suitable database.

BACKGROUND

Many different approaches are known to uniquely identify andauthenticate objects, including labeling and tagging strategies usingserial numbers, bar codes, holographic labels, RFID tags, and hiddenpatterns using security inks or special fibers. All of these methods canbe duplicated, and many add substantial costs to the production of thegoods sought to be protected. Physical labels and tags are alsothemselves at risk of being lost, stolen, or counterfeited.

Other known techniques for object identification or authentication relyon the comparison of bitmaps of images of objects. Referring now to FIG.8, an image of an object is captured and stored for reference. The wholeimage is stored, although it may be compressed for efficiency. When anew object is encountered, an image is captured of the new object andcompared to the original image, using XOR or similar algorithms. Ifthere are no (or only statistically insignificant) differences, theimages are declared a match and the object is authenticated. Further,FFT or similar transforms may be used to generate a “digital signature”of the image that can be used for comparison. See FIG. 9. However, as inthe previous case the same method is used, i.e. the resultant bitmappedimage is compared with another bitmapped image, and if the pixels matchthe object is authenticated. Such methods are disclosed in U.S. Pat. No.7,680,306 to Boutant, et al. Bitmapped techniques have limitations thatrender them unusable in many real world applications, such limitationsconsisting of, for example, inefficiencies due to file sizes,difficulties presented by variable lighting and orientation of images,and the inability to authenticate worn, damaged, or otherwise alteredobjects.

SUMMARY OF THE PRESENT DISCLOSURE

The following is a summary of the present disclosure in order to providea basic understanding of some features and context. This summary is notintended to identify key/critical elements of the disclosure or todelineate the scope of the disclosure. Its sole purpose is to presentsome concepts of the present disclosure in a simplified form as aprelude to a more detailed description that is presented later.

There are many known approaches to establishing or reestablishing theauthenticity of an object, including secure supply chains, expertassessment, and counterfeit detection. What is lacking, however, and isprovided by the current disclosure, is the ability to perform controlledauthentications based on the use of digital fingerprints and fingerprinttemplates for a variety of counterfeiting, conformity, andnon-conformity assessments.

In an embodiment, at least a portion of a physical object is scanned anda digital image of the object is generated from the scan. At least onesubset of the image, known as an “authentication region” is selected. Aplurality of feature vectors, arising from the physical structure of anobject, are extracted from certain locations of interest within anauthentication region and combined to generate a unique identifier or“digital fingerprint” for that object. The process of extracting featurevectors from locations of interest may be referred to as a “featureextraction.” The digital fingerprint may be registered in a database.

In a preferred embodiment, authentication regions for feature vectorextraction are automatically selected by a software process. Thesoftware process preferably selects a large number of authenticationregions for purposes of digital fingerprinting. An authentication regionmay be of interest because of a relatively high level of content.Content may comprise a gradient or vector, including a change in valueand/or direction.

In a preferred embodiment, each region of interest is identified, forinstance, as a circle by centroid location and radius. Within eachcircular region of interest, software may extract one or morefingerprint feature that defines relevant shapes within a correspondingcircular location of an image. Each fingerprint feature may be stored asa feature vector as illustrated below. A feature vector preferably is anarray of integer or floating point values describing a location ofinterest.

In order to authenticate an object, a suitable system may compare adigital fingerprint of an object to digital fingerprints previouslystored in a database, and, based on such a comparison, determine whetherthe object has been registered before, and, is thus authentic. Digitalfingerprint data may specify a set of features (also termed “locationsof interest”) which may be referenced in the creation of a “fingerprinttemplate” which is a template of certain locations of interest and/orattributes selected for authenticating a particular class of objects. Afingerprint template may, for example, identify the coordinates andshape of particular features. A fingerprint template may in someinstances be referred to as a feature template and the terms should bedeemed equivalent. A key advantage of the feature-based method is thatwhen an object is worn from handling or use, a feature-based system maystill identify the object as authentic, which may be impossible toaccomplish with the bitmapped approach.

Another aspect of the present disclosure relates to detectingcounterfeit or altered objects, for example, a forged driver's licenseor passport. In the case of some objects, such as forgeries, there maybe no original or source digital fingerprint available for comparison ormatching. In such cases—or in any cases where object comparisons beyondoriginal digital fingerprints are desired—an object may, for instance,be compared to digital fingerprints of known artifacts of counterfeit oraltered objects. In one example, a large number of counterfeit New YorkState driver's licenses might be obtained by law enforcement officialsin a raid or the like. Digital images of forged documents may beacquired and analyzed to generate digital fingerprints, as described inmore detail below. Feature vectors of typical features that occur in thecounterfeit licenses may be collected and stored in a database. Suchfeatures may include, for example, sharp, non-bleeding edges where aphotograph has been replaced or torn paper fibers where an erasureoccurred. The stored features from counterfeit licenses may be analyzedand stored as a reference set of fraudulent methods which may becompared to digital fingerprints of other licenses in an effort todetect forged licenses. In some embodiments, a count of fraud indicatormatches may be compared to an empirical threshold to determine andquantify a confidence that a certain document is forged, or not.

Further, the fingerprinting approach described below can be used todetermine how an object conforms to a predetermined specification, suchas the degree to which a manufactured object meets its manufacturedspecifications. Applications of the system include but are not limitedto object authentication, anti-counterfeiting, determining theprovenance of an object, and compliance with predetermined objectspecifications.

Additional features of this disclosure include: Centralized databasesstoring digital fingerprints of objects which enable enhanced security,rapid searching, and high reliability. According to one method, a partof the induction/authentication process is done remote from a facilityintended for this purpose. “In field” induction of an object may beconducted at almost any location using systems of the type describedbelow. The term “induction” is used in the sense of inducting an object(or set of objects) into a secure database system; that is, creating andstoring in the database system a special set of fingerprint data ordigital fingerprints that are unique to the object. Induction may bedone by a person, such as a technician, or it may be automated.Induction may be done clandestinely, so that the possessor of the objectis not made aware of the induction. Other parts of aninduction/authentication process may be done without the knowledge ofthe then-current possessors of the objects. In particular, when theinduction and/or authentication are not part of the current possessors'normal processes. The term “authentication” is used here to includetracking an object, with or without regard to its “genuineness” orprovenance.

Additional aspects and advantages of the present disclosure will beapparent from the following detailed description of preferredembodiments, which proceeds with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an example of an authentication region and fingerprinttemplate definition for a U.S. passport.

FIG. 2 is a simplified flow diagram of a process for digital fingerprintgeneration and registration.

FIG. 3 is a simplified flow diagram of a process for authentication of apreviously fingerprinted object.

FIG. 4 is a simplified flow diagram illustrating a process for objectinspection to detect evidence of counterfeits.

FIG. 5 is a simplified flow diagram illustrating an object inspectionprocess.

FIG. 6 is a simplified flow diagram illustrating a method for building adatabase for use in detecting forged or altered documents.

FIG. 7 is a simplified flow diagram illustrating a method for using adigital fingerprint of a suspect document to detect a potential forgeror alteration associated with the document.

FIG. 8 is a simplified diagram of a prior art bitmap comparison methodfor comparing images.

FIG. 9 is an example of a photograph and an image created by FastFourier Transform (FFT) of the image data.

FIG. 10 is a simple illustration of fingerprint feature extraction froman original digital image.

FIG. 11 is a simple illustration of fingerprint feature extraction froma comparison or candidate image.

FIG. 12 is a simple illustration of fingerprint feature comparison foridentifying or authenticating an object.

FIG. 13A shows an image of the numeral “3”, representing the first digitin a serial number of an “original” or known U.S. dollar bill.

FIG. 13B shows an image of the numeral “3”, representing the first digitin a serial number of a U.S. dollar bill to be authenticated.

FIG. 14A is an illustration of results of feature extraction showingselected locations of interest in the image of FIG. 13A.

FIG. 14B is an illustration of results of feature extraction showingselected locations of interest in the image of FIG. 13B.

FIG. 15A shows the same dollar bill image as in FIG. 13A, juxtaposedwith FIG. 15B for comparison.

FIG. 15B shows an image of the numeral “3” that has been damaged ordegraded.

FIG. 16A shows detail of two fingerprint feature locations on thenumeral 3.

FIG. 16B shows detail of the damaged bill with the correspondingfingerprint feature locations called out for comparison.

FIG. 17 is a simplified illustration of a rotational transformation inthe process of comparing digital fingerprints of two images.

FIG. 18 is a simplified flow diagram of an induction-authenticationprocess.

FIG. 19 is a simplified flow diagram of an in-field induction process toenable tracing an object.

FIG. 20 is a simplified flow diagram of a process utilizing controlledinduction and stealth authentication of an object, utilizing acentralized, secure, digital fingerprinting system.

FIG. 21 is a simplified flow diagram illustrating an example of aprocess for creating a digital fingerprint comprising feature vectorsfrom scan data.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

The methods described in this disclosure enable the identification of anobject without the need for attaching or associating physical tags ormaterials with the object. A system does this by creating a uniquedigital signature for the object, which is referred to as a digitalfingerprint. Digital fingerprinting utilizes the structure of theobject, including random and deliberate features created during themanufacturing or use of the object, to generate a unique digitalsignature for that object similar to the way in which a humanfingerprint references the friction ridges on a finger. Also, like ahuman fingerprint, the digital fingerprint can be stored and retrievedto identify objects when they are encountered at a later date.

Eliminating the need to add tags or any physical modifications to theobject offers a number of advantages to manufacturers, distributors,sellers, and owners of goods. Forgoing the addition of physicalidentifiers reduces the cost of manufacturing and is more secure thanphysical tagging. Moreover, physical identifiers can be lost, modified,stolen, duplicated, or counterfeited whereas digital fingerprintscannot.

Unlike prior art approaches that simply utilize a comparison of pixels,a system in accordance with the present disclosure utilizes theextraction of features to identify and authenticate objects. Featureextraction enables us to take a large amount of information and reduceit to a smaller set of data points that can be processed moreefficiently. For example, a large digital image that contains tens ofthousands of pixels may be reduced to just a few locations of interestthat can be used to identify an object. This reduced set of data iscalled a digital fingerprint. This digital fingerprint contains a set offingerprint features or locations of interest which are stored asfeature vectors. Feature vectors make image processing more efficientand reduce storage requirements as the entire image need not be storedin the database, only the feature vectors need to be stored. Examples offeature extraction algorithms include but are not limited to edgedetection, corner detection, blob detection, wavelet features, Gabor,gradient and steerable output filter histograms, scale-invariant featuretransformation, active contours, shape contexts, and parameterizedshapes.

While the most common applications of our system may be in theauthentication of manufactured goods and documents, the system isdesigned to be applicable to any object that can be identified,characterized, quality tested, or authenticated with a digitalfingerprint. These include but are not limited to mail pieces, parcels,art, coins, currency, precious metals, gems, jewelry, apparel,mechanical parts, consumer goods, integrated circuits, firearms,pharmaceuticals, and food and beverages. Here the term “system” is usedin a broad sense, including the methods of the present disclosure aswell as apparatus arranged to implement such methods.

Scanning

In an embodiment, an object is scanned and identified either at initialmanufacture or at the time of first contact with the system. This pointof identification is preferably done when the object is either in thepossession of its manufacturer, or has been transferred by secure meansto the current holder so that its legitimacy at point of identificationis adequately established. When such a process is impossible, as in theexample of artworks or old coins, the object may be fingerprinted afterthe object is authenticated by an expert while its provenance is stillsecure.

In this application, the term “scan” is used in the broadest sense,referring to any and all means of capturing an image or set of images,which may be in digital form or transformed into digital form. Imagesmay, for example, be two dimensional, three dimensional, or in the formof a video. Thus a “scan” may refer to an image (or digital data thatdefines an image) captured by a scanner, a camera, a specially-adaptedsensor array (such as CCD array), a microscope, a smartphone camera, avideo camera, an x-ray machine, a sonar, an ultrasound machine, amicrophone (or other instruments for converting sound waves intoelectrical energy variations), etc. Broadly, any device that can senseand capture either electromagnetic radiation or mechanical wave that hastraveled through an object or reflected off an object or any other meansto capture surface or internal structure of an object is a candidate tocreate a “scan” of the object. Various means to extract “fingerprints”or features from an object may be used; for example, through sound,physical structure, chemical composition, or many others. The remainderof this application will use terms like “image” but when doing so, thebroader uses of this technology should be implied. In other words,alternative means to extract “fingerprints” or features from an objectshould be considered equivalents within the scope of this disclosure.

Authentication Regions

Because digital fingerprinting works with many different types ofobjects, it may be helpful to define what regions of digital images ofobjects are to be used for the extraction of features for authenticationpurposes. The chosen regions may vary for different classes of objects.In some embodiments a chosen region may be the image of the entireobject; in other embodiments chosen regions may be one or moresub-regions of the image of the object.

For instance, in the case of a photograph, a digital image of the entirephotograph may be chosen for feature extraction. Each photograph isdifferent, and there may be unique feature information anywhere in aphotograph. In such a case, the authentication region may be the entirephotograph.

In some embodiments, multiple regions may be used for fingerprinting. Insome examples, there may be several regions where significant variationstake place among different similar objects that need to be distinguishedwhile, in the same objects, there may be regions of little significance.In other examples, a template (see FIG. 1) may be used to define regionsof interest, including elimination of regions of little interest.

In one embodiment, an object, such as a bank note, may be deemedauthenticated if a few small arbitrary regions scattered across thesurface are fingerprinted, possibly combined with one or morerecognitions of, for example, the contents of a region signifying thevalue of the bank note or one containing the bank note serial number. Insuch examples, the digital fingerprints of any region (along withsufficient additional information to determine the bank note value andits purported identity) may be considered sufficient to establish theauthenticity of the bill. In some embodiments, multiple fingerprintedregions may be referenced in cases where one or more region may beabsent from an object (through, for example, tearing) when, for example,a bank note is presented for authentication. In other embodiments,however, all regions of an object may need to be authenticated to ensurean object is both authentic and has not been altered.

In some cases, however, specific regions of an object may need to beauthenticated to ensure the object is both authentic and has not beenaltered. A passport provides an example. In the case of a passport, thefeatures chosen for authentication may be extracted from regionscontaining specific identification information such as the passportnumber, the recipient name, the recipient photo, etc. as illustrated inFIG. 1. In some examples, a user may define a fingerprint templatespecifying the regions whose alteration from the original wouldinvalidate the passport, such as the photo, identifying personal data,or other regions considered important by the user.

FIG. 1 illustrates one example of an authentication region and afingerprint template definition for a U.S. passport. In this figure,brace 101 refers to a simplified flow diagram of a process as follows:At process block 102, an object is scanned to generate an “originalimage”, that is, a digital image file or a digital data file in anysuitable format that is herein simply referred to as an “image”. Theoriginal image is illustrated as the data page spread of a U.S. passportbook, at block 150.

Next, the system processes the image data to determine an authenticationregion. In this example, the authentication region is the biographicdata page of the U.S. Passport, located in the lower portion of image150, identified by dashed box 154. Next, the process generates anauthentication image for feature extraction, block 106. Theauthentication image is illustrated at reference 156. Next, at block108, the process defines one or more locations of interest for featurevector extraction. The locations of interest in this example are, asshown in image 158 by dashed boxes 160, the surname, the given name, thepassport number, and the passport photo.

Finally, at block 110, the process 100 comprises creating a fingerprinttemplate 120. In this example, template 120 identifies an object class(U.S. Passport), defines an authentication region (for example, by X-Ycoordinates), and lists one or more locations of interest within thatauthentication region. In this instance, the list comprises passportnumber, photo, first name, and last name.

In some embodiments, an ability to define and store authenticationregions for classes of objects may offer benefits to a user. In someembodiments, it may be preferable to scan limited regions of objectsthan to scan entire objects. For instance, in the case of an article ofdesigner clothing, scanning a clothing label may be preferable toscanning an entire garment. Further, defining such regions may enabledetection of a partial alteration of the object.

Once an authentication region is defined, specific applications may becreated for different markets or classes of objects that may assistusers in locating and scanning suitable authentication regions. In someembodiments, for example when utilizing a mobile device, a location boxand crosshairs may automatically appear in the viewfinder of asmartphone camera application to assist a user in centering the cameraon an authentication region. The application may then automatically lockonto a region and complete a scan while the device is focused on anappropriate area.

In some embodiments, objects may have permanent labels or otheridentifying information attached to them. In addition to the objectsthemselves, these attachments may also be referenced as locations ofinterest for digital fingerprinting, particularly where the label orother identifying information becomes part of the object. In oneexample, a permanent label may be used as an authentication region forthe object to which it is affixed. In another example, a label may beused in conjunction with the object itself to create a fingerprint ofmultiple authentication regions referencing both a label and thesubstrate to which the label is affixed.

In one example, wine may be put into a glass bottle and a label affixedto the bottle. Since it is possible a label may be removed andre-applied elsewhere, merely using the label itself as theauthentication region may not be sufficient. In this case theauthentication region may be defined so as to include both the label andthe substrate the label is attached to—in this case some portion of theglass bottle. This “label and substrate” approach may be useful indefining authentication regions for many types of objects, such asvarious types of goods and associated packaging. In other instances,authentication may reveal changes in the relative positions of someauthentication regions such as in cases where a label has been movedfrom its original position, which may be an indication of tampering orcounterfeiting. If an object has “tamper-proof” packaging, this may alsobe included in the authentication region.

In some cases, multiple authentication regions may be selected forfeature extractions. For example, in one embodiment, features may beextracted from two different parts of a firearm. Both features may matchthe original firearm but since it is possible that both parts may havebeen removed from the original firearm and affixed to a weapon ofdifferent quality, it may also be useful to determine whether therelative positions of the features have changed. In other words it maybe helpful to determine that the distance (or other characteristics)between Part A's authentication region and Part B's authenticationregion remains consistent with the original feature extraction. If thepositions of Parts A and B are found to be consistent to the relativepositions of the original authentication regions, the firearm may beauthenticated. Specifications of this type may be stored with or as partof a digital fingerprint of an object.

Once a digital fingerprint of an object is acquired, a characterizationof the object and corresponding fingerprint may be recorded in adatabase. For example, in some embodiments, a fingerprint may compriseone or more feature vectors. In some applications, the database may besecure. In some embodiments, a unique identifier or ID may be assignedto an object, for example in cases where an ID may be a convenient indexin an application. However, an ID is not required since a digitalfingerprint itself may serve as a key for searching a database. In otherwords, by identifying an object by the unique features andcharacteristics of the object itself, arbitrary identifiers, labels,tags, etc. are rendered unnecessary.

Fingerprint Template Definition

In an embodiment, when a new type or class of object is being scannedinto a system for the first time, the system can create a fingerprinttemplate (as shown in FIG. 1) that can be used to control subsequentauthentication operations for that class of objects. This template maybe created either automatically by the system or by a human-assistedprocess.

A fingerprint template is not required for the system to authenticate anobject, as the system can automatically extract features and create adigital fingerprint of an object without it. However, the presence of atemplate may optimize the authentication process and add additionalfunctionality to the system.

TABLE 1 Example Fingerprint Template. CLASS: [Description of the object]United States Passport AUTHENTICATION REGION: [Description of theauthentication regions for the object] Region 1: (x1, y1, z1), (x2, y2,z2) . . . Region n REGION MATCH LIST [List of the regions that arerequired to match to identify an object] Region List: 1 . . . nFEATURES: [Key features of the object] Feature 1: Passport NumberFeature 2: Photo Feature 3: First Name Feature 4: Last Name . . .Feature n METHODS: [Programs that can be run on features of an object]Feature 2: Photo Method 1: [checkphoto.exe] Check for uneven edgesindicating photo substitution . . . Method n Feature n Method nADDITIONAL DATA [Additional data associated with the object] Data 1:example data . . . Data n

The uses of the fingerprint template include but are not limited todetermining the regions of interest on an object, the methods ofextracting fingerprinting and other information from those regions ofinterest, and methods for comparing such features at different points intime. The name “fingerprint template” is not important; other data withsimilar functionality (but a different name) should be consideredequivalent.

In an embodiment, four different but related uses for this technologyare particularly in view in the present disclosure. These areillustrative but are not intended to be limiting of the scope of thedisclosure. These applications may be classified broadly as (1)authentication of a previously scanned original, (2) detection ofalteration of a previously scanned original, (3) detection of acounterfeit object without benefit of an original, and (4) assessing thedegree to which an object conforms with a predetermined specification,such as a manufacturing specification.

In example (1), an object is fingerprinted preferably during thecreation process (or at any time when its provenance may be sufficientlyascertained) or at a point where an expert has determined itsauthenticity. Subsequently, the object is later re-fingerprinted, andthe two sets of fingerprints are compared to establish authenticity ofthe object. The fingerprints may be generated by extracting a singlefingerprint from the entire object or by extracting multiple sets offeatures from multiple authentication regions. Fingerprinting may alsoinvolve reading or otherwise detecting a name, number, or otheridentifying characteristics of the object using optical characterrecognition or other means which may be used to expedite or facilitate acomparison with other fingerprints. For instance, in cases wheremanufacturing (or other object) databases use serial numbers or otherreadable identifiers, such identifiers may be utilized to directlyaccess the database record for the object and compare its digitalfingerprint to the original that was previously stored, rather thansearching an entire digital fingerprinting database for a match.

In case (2), a fingerprinted object is compared, region by region, witha digital fingerprint of an original object to detect low or nonexistentmatching of the fingerprint features from those regions. While case (1)is designed to determine whether the original object is now present,case (2) is designed to detect whether the original object has beenaltered and, if so, how it has been altered. In some embodiments,authentication regions having poor or no matching fingerprint featureswill be presumed to have been altered.

In case (3), an object may not have been fingerprinted while itsprovenance was sufficiently ascertainable. One example would be bills orpassports created prior to initiating the use of a digitalfingerprinting system. In such examples, digital fingerprints of certainregions of interest on an object may be compared with digitalfingerprints from known, or suspected, counterfeit objects or with boththose and fingerprints of properly authenticated objects. In oneexample, a photograph may be spuriously added to a passport and, as anartifact of the counterfeiting, the edge of the added photo may tend tobe sharper than an edge of an original, unaltered, photograph. In such acase, fingerprint characteristics of known authentic passports and thoseof passports that are known (or suspected to) have been altered bychanging a photograph may be compared with the passport being inspectedto estimate whether the passport exhibits indications of alteration.

FIG. 6 is a simplified flow diagram of a process 600 for building adatabase for use in detecting counterfeit or suspected counterfeit(forged or altered) objects, herein a document. At process block 602,digital image data is acquired of a known or suspected forged or altereddocument. Next, features are extracted from the image data at block 604.Continuing, at block 606, a digital fingerprint is created based on theextracted features.

The digital fingerprint data is stored in a database record, block 608.Further, the record (digital fingerprint) is designated in the databaseas having features tending to evidence of a known or suspected forgery,block 610. The basic process may be repeated, loop 650, to acquire moreimages, and more features, to build the database.

In case (4), the question of authenticity or alteration is not at issue.Instead, the fingerprinting process is used to assess how an objectconforms to a predetermined specification, such as a manufacturingspecification. In such embodiments, the comparison of fingerprintfeatures of a target object is made with reference to certainpredetermined features of a reference object, preferably an idealreference object. The reference object may exist as a physical exampleor may be a programmatic prescription. In the case of programmaticallyprescribed fingerprint features the fingerprint features will begenerated manually or by a program rather than by scanning an originalobject.

A fingerprint template may contain a variety of information related to aclass of objects. In an embodiment, a fingerprint template preferablyincludes one or more authentication regions for a class of objects—whichauthentication regions are utilized for matching—and a list of keyfeatures that are preferably used in authenticating an object of thatclass.

In some embodiments, a fingerprint template may define methods forexamining an object for signs of modification or counterfeiting. Forinstance, every time a passport is scanned by a system, a program may berun to examine the passport for signs of alteration. If the passport wasfingerprinted at an earlier point in time, for example at creation, thefingerprints previously extracted from one or more regions may becompared to fingerprints from corresponding regions at the time thepassport is presented for authentication. If the passport was notfingerprinted at creation, the fingerprint template may be used toexamine the passport for counterfeiting artifacts, such as sharp ornon-bleeding edges that may indicate that a photograph has been replacedor torn paper fibers that may indicate where an erasure has occurred. Inaddition to the examples discussed above, the fingerprint template isdesigned to be extensible and may store any additional data that isrelated to the object.

Digital Fingerprint Generation

In an embodiment, once an object has been scanned and at least oneauthentication region has been identified, the digital image, which willbe used to create the unique digital fingerprint for the object, isgenerated. The digital image (or set of images) provides the sourceinformation for the feature extraction process.

In the present disclosure, a digital fingerprinting feature is definedas a feature or a location of interest in an object, which feature isinherent to the object itself. In some embodiments, features preferablyare a result of a manufacturing process, other external processes, or ofany random, pseudo-random, or deliberate process or force, such as use.To give one example, gemstones have a crystal pattern which provides anidentifying feature set. Every gemstone is unique and every gem stonehas a series of random flaws in its crystal structure. This pattern ofrandom flaws may be used for the extraction of feature vectors foridentification and authentication.

In the present disclosure, a “feature” is not necessarily concerned withreading or recognizing meaningful content, for example by using methodslike optical character recognition. A digital fingerprint of an objectmay capture both features of the object and features of any identifiersthat are affixed or attached to the object. Feature vectors extractedfrom authentication regions located on an affixed identifier are basedon the substances of which the identifier is physically comprised ratherthan the information (preferably alphanumeric) that is intended to becommunicated by the identifier. For instance, in the case of a winebottle, features may be captured from the bottle and from a labelaffixed to the bottle. If the label includes a standard UPC bar code,the paper of the label and the ink pattern of the bar code may be usedto extract a feature vector without reading the alphanumeric informationreflected by the bar code. An identifier, such as a UPC bar code printconsisting of lines and numbers, has no greater significance in thegeneration and use of a feature vector than a set of randomly printedlines and numbers.

Although reading identifier information is not necessary for digitalfingerprinting, in some embodiments, where a user desires to capture orstore identifier information (such as a name, serial number, or a barcode) in an association with an object, the system may allow the user tocapture such information and store it in the digital fingerprint.Identifier information may, for example, be read and stored by utilizingtechniques such as optical character recognition, and may be used tofacilitate digital fingerprint comparisons. In some cases, serialnumbers may be used as the primary index into a database that may alsocontain digital fingerprints. There may be practical reasons forreferencing serial numbers in relations to digital fingerprints. In oneexample, a user is seeking determine whether a bank note is a match witha particular original. In this case, the user may be able to expeditethe comparison by referencing the bank note serial number as an indexinto the digital fingerprinting database rather than iterating through alarge quantity of fingerprints. In these types of cases, the indexrecognition may speed up the comparison process but it is not essentialto it.

Once a suitable digital fingerprint of an object is generated thedigital fingerprint may be stored or registered in a database. Forexample, in some embodiments, the digital fingerprint may comprise oneor more fingerprint features which are stored as feature vectors. Thedatabase should preferably be secure. In some embodiments, a uniqueidentifier, such as a serial number, may also be assigned to an objectto serve, for example, as a convenient index. However, assigning aunique identifier is not essential as a digital fingerprint may itselfserve as a key for searching a database independent of any addition of aunique identifier. In other words, since a digital fingerprint of anobject identifies the object by the unique features and characteristicsof the object itself the digital fingerprint renders unnecessary the useof arbitrary identifiers such as serial numbers or other labels andtags, etc.

FIG. 2 is a simplified flow diagram of a process 200 for digitalfingerprint generation and registration. In this case, the processbegins with scanning an object, block 202. An image 250 is acquired; inthis illustration a U.S. passport is used. The next step is to identifyor generate an authentication region, block 204. For example, theauthentication region may be example 252. The authentication region maybe identified, as discussed above, from a fingerprint template (seeTable 1, supra). Next an object class of the object is determined, block206. The result is used to check a database for a corresponding objectclass template, decision 208. If there is no matching template, theprocess proceeds to extract features, block 210, without the aid of atemplate. A digital fingerprint is created based on the extractedfeature, block 212, and the digital fingerprint is stored in a databaseat 220.

Alternatively, if a matching object class template is found at decision208, the process continues to extract features, block 222, utilizing theobject class template 223 to identify and locate the features. Next, adigital fingerprint is generated from the resulting feature data, block224, and stored in a database 230.

Example Authentication and Inspection Processes

In an embodiment, an object is scanned and an image is generated. Thesteps that follow depend on the operation to be performed. Severalillustrative example cases are discussed below.

Case 1: For authentication of a previously fingerprinted object, thefollowing steps may be followed (see FIG. 3, discussed below):

-   -   1. One or more authentication regions are either determined        automatically by a system, or by utilizing the authentication        region definitions stored in a fingerprint template.    -   2. Relevant features are extracted from each authentication        region and a digital fingerprint is created. Feature extractions        preferably will be in the form of feature vectors, but other        data structures may be used as appropriate.    -   3. Optionally, a unique identifier such as a serial number may        be extracted and stored to augment subsequent search and        identification functions.    -   4. The digital fingerprint of the object to be authenticated is        compared to digital fingerprints stored in a database.    -   5. The system reports whether the object matches one of the        digital fingerprints stored in the database.    -   6. The system may store the digital fingerprint of the object to        be authenticated in the database along with the results of the        authentication process. Normally only the extracted features        will be stored in the database, but the authentication image        and/or the original image may be stored in the database, for        example for archival or audit purposes.

FIG. 3 illustrates such a process 300 in diagrammatic form. Beginning atstart block 302, the process scans an object and creates anauthentication image, block 304. The image is represented at 350, usinga passport as an example. Features are extracted, block 306, andoptionally, other information, such as a serial number or similar IDnumber, preferably unique, may be extracted as well, block 310.

The extracted data is processed to create a digital fingerprint, bloc312. A database 320 may be queried for a matching fingerprint, block314. A “match” may be defined by a probability or similarity metric.Results of the database query may be reported to a user, block 322.Finally, a new digital fingerprint may be stored into the database 320,shown at process block 330.

Case 2: For inspection of specific features of a previouslyfingerprinted object to determine whether they have been altered, thesteps are similar to Case 1, but the process is aimed at detection ofalterations rather than authentication of the object:

-   -   1. One or more authentication regions are either determined        automatically by the system, or by utilizing the authentication        region definitions stored in a fingerprint template.    -   2. The features to be inspected are extracted from the        authentication region and the digital fingerprint is created.        The features extracted may be in the form of feature vectors for        the features to be inspected but other data structures may be        used as appropriate.    -   3. Optionally, other information, for example a unique        identifier such as a serial number may be extracted and stored        to be used to augment subsequent search and identification        functions.    -   4. The digital fingerprint of the features to be inspected for        alteration is compared to the fingerprint of the corresponding        features from the original object stored in the database.    -   5. The system reports whether the object has been altered; i.e.        whether the digital fingerprint of the features to be inspected        match those previously stored in the database from the original        object.    -   6. The system may then store the digital fingerprint of the        features to be inspected in the database along with the results        of the inspection process. Normally only the features will be        stored in the database, but the authentication image and/or the        original image may be stored in the database for archival or        audit purposes.

Case 3: For inspection of specific features of an object that has notbeen previously fingerprinted in order to determine whether the featureshave been altered, the following steps may be followed, referring now toFIG. 4.

A system scans an object, at block 404, and creates an authenticationimage 450 that includes at least one authentication region. Eachauthentication region may be determined automatically by the system, orby utilizing the authentication region definitions defined in a storedfingerprint template 406, as noted supra. Either way, the process nextextracts features from the authentication region(s), block 408, and adigital fingerprint is generated. The extracted features preferably willbe in the form of feature vectors, but other data structures may be usedas appropriate.

The features of the object are then analyzed, block 420, and examinedfor attributes indicative of counterfeiting, block 402. Methods may beapplied to the features by running programs that are listed in thefingerprint template 406 that match features with signs ofcounterfeiting. Features may also be statistically compared to featuresof other objects of the same class that are stored in the database, forexample by using Bayesian algorithms or other methods to find suspectvariations that the standard methods may not catch. Optionally, otherinformation, such as a serial number or similar ID may be extracted,block 410.

The system preferably reports whether the object shows signs ofalteration or counterfeiting, block 422. The system may then store thedigital fingerprint of the object to be inspected, block 424, in thedatabase 430 along with the results of the inspection process.Preferably only the extracted features will be stored in the database,but the authentication image and/or the original image may be stored inthe database, for example for archival or audit purposes.

Case 4: For inspection of an object to estimate how it conforms to oneor more predetermined specifications—for example whether an object wasmanufactured in conformance with a manufacturer's specification, thefollowing steps may be followed; referring now to FIG. 5. One or moreauthentication regions for an object 502 are determined by utilizing theauthentication region definitions stored in a fingerprint template 506.In this illustration, the object 502 is a U.S. $100 bill. Scanning andgeneration of an authentication image are indicated at process block504.

Target features are extracted from authentication regions, block 508,and a digital fingerprint is created (not shown). The feature extractionpreferably will be in the form of feature vectors for the targetfeatures, but other data structures may be used as appropriate.Optionally, other information, including a unique identifier such as aserial number may be extracted, block 510, and stored to be used toaugment subsequent search and identification functions.

Next, the digital fingerprint of the target features is analyzed, block520, and compared to a fingerprint of predetermined features orspecifications at block 521. In some embodiments, the predeterminedfeatures used for comparison are obtained by digitally fingerprinting areference object (such as an ideal manufactured object), stored in adatabase at block 521. In other embodiments, the digital fingerprint ofthe object, and more specifically the feature vectors extracted from thetarget features, may be compared to reference feature vectors based onpredetermined specifications that are stored in a database at 521. Thistype of comparison seeks to estimate the conformity of the object butmay not indicate provenance

The system reports, at block 522, whether the object meetsspecifications; i.e. whether the digital fingerprint of the targetfeatures sufficiently matches those stored in the database from thereference object (or reference specifications). The system may thenstore the digital fingerprint of the target features in the database530, process block 524, along with the results of the inspectionprocess. Preferably only the extracted target features will be stored inthe database, but the inspection image and/or the original image may bestored in the database, for example for archival or audit purposes.

In all of the above cases, features may be extracted from images ofobjects scanned under variable conditions, such as different lightingconditions. Therefore, it is unlikely two different scans will producecompletely identical digital fingerprints. In a preferred embodiment,the system is arranged to look up and match objects in the database whenthere is a “near miss.” For example, two feature vectors [0, 1, 5, 5, 6,8] and [0, 1, 6, 5, 6, 8] are not identical but by applying anappropriate difference metric the system can determine that they areclose enough to say with a degree of certainty that they are from thesame object that has been seen before. One example would be to calculateEuclidean distance between the two vectors in multi-dimensional space,and compare the result to a threshold value. This is similar to theanalysis of human fingerprints. Each fingerprint taken is slightlydifferent, but the identification of key features allows a statisticalmatch with a high degree of certainty.

FIG. 7 is a simplified flow diagram illustrating a method 700 for usinga digital fingerprint of a suspect document to detect a potentialforgery or alteration associated with the document. First, image data isacquired of a suspect document at block 702. Then the process extractsfeatures from a selected region of the document, block 704. Theextracted features are used to form a digital fingerprint, block 706.Next, the digital fingerprint or the extracted features are used to forma query, and the query is used to access a forgery and/or alterationdatabase in search of a matching record, block 708. If a matching recordis returned, at decision block 710, then the system may report apotential forgery or alteration associated with the suspect document,block 712. Optionally, multiple results may be combined in reaching aconclusion, block 720, where such are available.

Referring again to decision block 710, if no match is returned (i.e. norecord matches the query criteria within a selected tolerance orconfidence) the process optionally may be repeated, block 714, forcomparison to additional database records. In other words, the databasesearch may be expanded, see loop 718. Again, multiple query results maybe combined. Further, the entire process, defined by loop 730, may berepeated for inspecting and analyzing additional or different regions ofthe document, block 722. As discussed earlier, multiple regions ofinterest may be defined. Terminal conditions, not shown, may beimplemented. Although this illustration uses a document as an examplethe process may be applied to any physical object.

FIG. 10 is an illustration of an example of feature extraction from adigital image. The original image data, on the left, is searched by anyof various software processes to detect and locate a feature in theimage. In this case, the important shape in this image is a square, andthe process extracts that feature, as shown in the middle of the figure,with “1” pixel values. (Real-world implementations preferably will havegreater than one-bit pixel values.) Next, the extracted fingerprintfeature may preferably be stored as a feature vector—as illustrated onthe right side of the figure. A feature vector is an n-dimensionalvector of numerical values that represent the shape.

In this instance, the features alone may be stored and not necessarilythe entire image. In fact, after feature extraction the original imagemay be discarded by the user, which may offer advantages with respect toreduced storage requirements. Typical algorithms used for extractingfeatures include, but are not limited to, edge detection, cornerdetection, blob detection, wavelet features, Gabor, gradient andsteerable output filter histograms, scale-invariant featuretransformation, active contours, shape contexts, and parameterizedshapes.

FIG. 11 illustrates the same process for accessing a comparison orcandidate image, extracting features from that image and storing each ofthem as a feature vector. Note that the example above is presentedsolely for the purpose of explanation—actual implementations will vary.For instance, many shapes may be parameterized and stored even moreefficiently. Instead of storing all the pixels along the boundaries, thesquare in FIG. 10 might simply be stored as the lower left and upperright corner points ((x1, y1), (x2, y2)). Similarly, a circle might bestored with just the center point and radius. Feature vectors may storea wide variety of n-dimensional representations such as point, lines,polylines, edges, ridges, histograms, and many others.

Once features are extracted from the original image and the candidateimage, the features can be compared directly to determine if there is amatch. Typical algorithms for comparing features include, but are notlimited to, nearest neighbor, hashing, indexing feature sets with visualvocabularies, support vector machines, multilayer perceptron, and randomforests and ferns. An example of comparison of feature vectors isillustrated in FIG. 12, resulting in a match.

FIG. 13A illustrates an image of the numeral “3” representing a numberprinted on an “original” or known U.S. dollar bill. The bill may havebeen fingerprinted, for example, at the time of manufacture or publicrelease, as described herein, or otherwise sufficiently authenticatedfor use as a reference. As noted below, fingerprint databases ofcurrency and the like may be secured. Such databases preferably excluderaw image data. This image, on the order of about 40× magnification,shows a number of distinctive features visible to the naked eye.

FIG. 13B illustrates an image of a number printed on a second or unknownU.S. dollar bill. The second bill may be fingerprinted using the sameprocess, and then the resulting digital fingerprints, i.e., therespective fingerprint feature vectors, may be compared as furtherexplained below, to determine whether or not the second bill is in factthe same one as the first bill. The comparison may take place eventhough the bill may have changed from wear and tear.

FIG. 14A is a simplified illustration of the results of featureextraction applied to the numeral 3 of FIG. 13A. In this figure, onlythe ends of the numeral are shown. Two areas of interest are called outby circles 1710 and 1750. Below is a discussion on how these areas maybe selected in an image. Fingerprint feature extraction is applied toeach of the circular regions. The results for each location are storedas fingerprint feature vectors. A collection of feature vectors, say forlocation 1750, may be stored as a feature vector array. FIG. 14B is asimplified illustration of the results of feature extraction applied tothe numeral 3 of FIG. 13B. The same fingerprinting process is applied tothis image. The same locations of interest as in FIG. 14A are labeled1720 and 1760, respectively. The stored features (from the originalobject) are compared with the features extracted from the new object. Ifthe features of the original object are not encountered in the secondobject, the object are not a match.

One advantage of the feature-based method is that when an object is wornfrom handling or use (even very worn), a system may still identify theobject as original, which may be impossible with the bitmapped approach.FIG. 15A shows a numeral from the same dollar bill image as in FIG. 13A,juxtaposed with FIG. 15B for comparison. FIG. 15B shows the numeral onthe same bill after the bill has been subjected to washing in a washingmachine, perhaps as a result of being left in the pocket of a piece ofclothing.

In FIG. 15B, the image (or, rather, the dollar bill) has been degraded;there is significant loss of ink and destruction of the paper surface inmultiple locations. A bitmapped approach to matching would likely failto match these two figures due to the large number of pixels that arenow different, as relatively few of the pixels remain the same as theoriginal.

FIG. 16A shows the detail of two fingerprint feature locations asbefore, 1610 and 1650. FIG. 16B shows detail of the damaged bill withthe corresponding locations called out as 1620 and 1660, respectively. Acomparison between the similarities of area 1610 to area 1620 and ofarea 1650 to area 1660 illustrates how a comparison of the correspondingfingerprint feature vectors would be adequate to result in a match. Inpractice, a much larger number of features would be used.

The image of the damaged bill is analyzed by a processor. The processoraccesses a database of previously stored fingerprint data. If the dollarbill serial number is legible (by eye or machine), the record for thecorresponding bill may be accessed from the datastore using the serialnumber as an index. Similarly, if any portion of the serial number islegible, the search for a matching record can be narrowed on that basis.Either way, a candidate record, containing a set of stored regions ofinterest may be compared to the image of the damaged bill.

As explained above, in addition to being able to recognize a wornobject, the feature-based approach is able to address other externalproblems such as rotated images. This is especially important in asystem where an unsophisticated user, such as a retail customer, may bescanning an object to be authenticated. In such cases, external factorslike lighting and rotation may not be under the system operator'scontrol.

Referring now to FIG. 17, which shows the original image on the leftside, with a small set of fingerprint features marked as small diamondshapes. This is merely a callout symbol for illustration. In someembodiments, as noted, preferably circular areas are used. For eachfeature (preferably identified in the database record), a search isconducted of the suspect image on the right side of FIG. 17 (or aportion of it) for a matching feature. The position may not matchexactly, due to “stretch”, an effective difference in magnification,and/or due to rotation of the image, or due to other circumstances.Although it may not match locations literally; a mathematicaltransformation may be defined that maps one image to the other, therebyaccounting for rotation and stretch as appropriate. Thus, a boundingrectangle A indicated by the box in the left side image may be mapped toa quadrilateral, indicated by the line B in the right side image.

Once an appropriate transformation is found, further matching may bedone to increase the level of confidence of the match, if desired. Insome embodiments, a number of matches on the order of tens or hundredsof match points may be considered sufficient. The number of non-matchpoints also should be taken into account. That number should preferablybe relatively low, but it may be non-zero due to random dirt, system“noise”, and other circumstances. Preferably, the allowed mapping ortransformation should be restricted depending on the type of objectunder inspection. For instance, some objects may be inflexible, whichmay restrict the possible deformations of the object.

Summarizing the imaging requirements for a typical fingerprintingsystem, for example for inspecting documents, the system preferablyshould provide sufficient imaging capability to show invariant features.Particulars will depend on the regions used for authentication. For manyapplications, 10-fold magnification may be adequate. For ink bleeds onpassports, bills, and other high-value authentication, 40-foldmagnification may likely be sufficient. In preferred embodiments, thesoftware should implement a flexible response to accommodatemisalignment (rotation), misorientation, and scale changes. Colorimaging and analysis is generally not required for using the processesdescribed above, but may be used in some cases.

Hardware and Software

Most of the equipment discussed above comprises hardware and associatedsoftware. For example, a typical portable device is likely to includeone or more processors and software executable on those processors tocarry out the operations described. The term “software” is used hereinin its commonly understood sense to refer to programs or routines(subroutines, objects, plug-ins, etc.), as well as data, usable by amachine or processor. As is well known, computer programs generallycomprise instructions that are stored in machine-readable orcomputer-readable storage media. Some embodiments of the presentdisclosure may include executable programs or instructions that arestored in machine-readable or computer-readable storage media, such as adigital memory. It is not to be inferred that a “computer” in theconventional sense is required in any particular embodiment. Forexample, various processors, embedded or otherwise, may be used inequipment such as the components described herein.

Memory for storing software is well known. In some embodiments, memoryassociated with a given processor may be stored in the same physicaldevice as the processor (e.g. “on-board” memory); for example, RAM orFLASH memory disposed within an integrated circuit microprocessor or thelike. In other examples, the memory comprises an independent device,such as an external disk drive, storage array, or portable FLASH keyfob. In such cases, the memory becomes “associated” with the digitalprocessor when the two are operatively coupled together, or incommunication with each other, for example by an I/O port, networkconnection, etc. such that the processor can read a file stored on thememory. Associated memory may be “read only” by design (ROM) or byvirtue of permission settings, or not. Other examples include, but arenot limited to, WORM, EPROM, EEPROM, FLASH, etc. Such technologies areoften implemented in solid state semiconductor devices. Other memoriesmay comprise moving parts, such as a conventional rotating disk drive.All such memories are “machine readable” or “computer-readable” and maybe used to store executable instructions for implementing the functionsdescribed herein.

A “software product” refers to a memory device in which a series ofexecutable instructions are stored in a machine-readable form so that asuitable machine or processor, with appropriate access to the softwareproduct, can execute the instructions to carry out a process implementedby the instructions. Software products are sometimes used to distributesoftware. Any type of machine-readable memory, including withoutlimitation those summarized above, may be used to make a softwareproduct. It is also known that software can be distributed viaelectronic transmission (“download”), in which case there typically willbe a corresponding software product at the transmitting end of thetransmission, or the receiving end, or both.

Integration with Bill Processing Equipment

The creation of fingerprint data is proposed at the U.S. Treasury or anyother producer and/or printer of negotiable bills or notes. Preferably,such a system utilizes random, microscopic features unique to eachbill's paper and printing. For example, the system may extract featuresfrom unpublished locations on the bills. In other words, the specificlocations used for authentication are maintained in secrecy. A “featuretemplate” may be used, and the locations for feature extraction may bestored in the feature template, which in turn is stored in a securedatabase. The extracted features may be converted into encrypted featurevectors and associated in a data store with the corresponding billserial number (the serial number having been readily captured by thesame scanner or a separate one). In this way, a protected database maybe created that is addressable or searchable by serial number or featurevector, but only by authorized users. A “user” may be a machine withelectronic access to the database.

Equipment is known for stacking, counting, and “strapping” paper moneynotes. A “strap” is a package of 100 notes, held together by a singlepaper band, as required for deposit by U.S. Federal Reserve rules.Note-handling equipment may be modified to include a digital scanner,for example an optical scanner, to capture images of each bill or noteas it is processed. The scanner may be coupled to a suitable processor,as explained above, for storing the captured images, for processing theimages to authenticate them, and/or to detect counterfeit objects.

Preferably, such a system is granted access to the protected databasethat is searchable by serial number or digital fingerprint. It may thenlook up each bill scanned, and compare features of the digital image tothe digital fingerprint stored in protected database for thecorresponding serial number. This process may be done by batch or inreal time or near real time. The comparison, as further described above,may provide a confidence metric, a simple yes/no (authentic/counterfeit)result for each note, or a different suitable metric. It may identify acounterfeit note by serial number, but also by sequence number tofacilitate locating the bill (“the 28^(th) bill in the strap #218”). Inthis way, a bank or other institution can detect counterfeit notes in atimely manner.

In another embodiment, a scanner, which may be portable and optionallywireless, may be made available at a bank teller station for the tellerto authenticate bills presented for deposit or exchange. Further, such asystem may be installed at an ATM machine to automatically authenticatebills presented for deposit. The ATM may be programmed to accept bothauthenticated and counterfeit or suspected counterfeit bills to get them“off the street” but flag them as counterfeit or suspect.

The term “note” is commonly used in the U.K. with regard to paper money,while the term “bill” is more common in the U.S. The terms are usedinterchangeably herein. Not to be confused with U.S. Treasury “bills”and “notes” which are not currency but debt instruments. That said, thepresent disclosures herein are applicable to those as well as currency,although nowadays such things are mainly processed by electronic “bookentries” rather than paper documents. Older U.S. Savings Bonds, and anyother bearer instruments in any country, can all be authenticated byvarious embodiments of the present disclosure.

Induction and Authentication

FIG. 18 is a simplified diagram illustrating the concepts of inductionand authentication. The term “induction” is used in a general manner torefer to entering an object or a set of objects into an electronicsystem for subsequently identifying, tracking, or authenticating theobject, or for other operations. The object itself is not entered intothe system in a physical sense; rather, induction refers to creating andentering information into a memory or datastore from which it can laterbe searched, interrogated, retrieved, or utilized in other kinds ofdatabase operations.

In FIG. 18, induction 1802 thus may refer to a process that includescapturing an image of an object (or part of an object), processing theimage to extract descriptive data, storing the extracted data, or any orall of these operations. The inducted object, represented by a cube1804, then leaves the induction site, and proceeds in time and spacealong a path 1806. Induction may be done at the point of creation ormanufacture of the object, or at any subsequent point in time. In somecases, induction may be done clandestinely, meaning without theknowledge of the person or entity currently having possession of anobject. The term “possession” is used in the broadest sense to include,for example, actual physical possession, as well as control—for example,having they key to a secure physical storage where an object is kept.

After induction, the object 1804 may encounter wear and tear, andotherwise may change, intentionally or not, in ways that may not beknown a priori, represented by the question mark 1808. The originalobject 1804 may even in fact be lost or stolen after induction and acounterfeit may be introduced. Along path 1809, an object 1810 may bepresented for authentication, represented by block 1820. Below aredescribed some additional scenarios and use cases for the authenticationtechnology described herein, and what may be done under the broadheading of “authentication”. Under many circumstances, induction,authentication, or both may be done remotely by use of technology suchas drones or by other covert means. In one example, an agent may take aphotograph of an object with a smartphone, without the knowledge orconsent of the possessor of the object, and the resulting image may beutilized for induction and/or authentication as described herein.

More specifically, in some embodiments, some part of theinduction/authentication process may be done remote from a facilityintended for that purpose. In addition, some part of theinduction/authentication process may be accomplished without theknowledge of the then-current possessor of an object. In particular, theinduction and/or authentication are not part of the current possessors'normal processes. These two criteria are not essential for the presentdisclosure, but are generally representative of some applications.

FIG. 19 is a simplified flow diagram illustrating several differentscenarios to enable tracing an object. In this example, beginning atblock 1902, an object 1904 is of interest, but its provenance may beunknown. The object may be counterfeit or a genuine article. Moreover,the object may be stolen or dangerous in the wrong hands. The object'sbackground, legitimacy, or provenance is not a requirement in thisscenario. In this process, the object 1904 may be inducted at virtuallyany time and place where the object (or its digital image data) may befound, indicated by dashed lines 1933A and 1933B, using a fieldinduction machine 1920. This process may be overt or clandestine. Afield induction machine 1920 may comprise a scanner, camera, or thelike, as discussed above, to capture an image of the object 1904 or apart of it. Certain parts of an object may be selected for imaging basedon the type of object. The machine 1920 may comprise a processor forprocessing the image data to form digital fingerprints as describedabove. In other embodiments, the machine may have communicationscapability to send the raw image data for processing elsewhere—or it maystore the image for later use. In some embodiments the data is processedto extract features from the image data, as explained in detail earlier,and the extracted feature information is stored in the form of featurevectors, preferably in a secure datastore.

The field induction machine 1920 may be controlled remotely or it mayact autonomously. The machine may receive instructions on “where tolook” for features in captured image data. The “where to look”information may be in the form of a feature template. Feature templateinformation may be pre-programmed into the field induction machine, orit may be downloaded from a remote source. In another embodiment, thefield induction machine 1920 may itself process image data to findregions of interest and extract feature information from those regionsof interest. The field induction machine may send the extracted featureinformation, preferably in the form of feature vectors, to a securedatabase server 1930. In some embodiments, the field induction machine1920 may send raw image data to the server 1930, which in turn willprocess the data to extract feature information, and store thatinformation in a datastore 1932.

Previous patents have dealt with induction/authentication of objectsknown to be genuine at the point of induction (e.g. machine parts beforethey leave the manufacturer and enter the supply chain) or objects whichhave some form of “second provenance” such as induction in concert withan expert or authority, for example a wine or art expert, authenticatingthe object. Here, legitimacy of the object is not in view. For example,a device such as field induction machine 1920 could be used in the fieldby a third party, such as a law enforcement agency, to surreptitiouslyinduct, for example, counterfeit or suspected counterfeit currency at ornear the point of production, with the “authentication” to be performedlater at a bank where the counterfeit is recognized. This section of thedisclosure is not primarily used to authenticate an object as genuine,but rather to provide traceability to determine where and how a possiblycounterfeit or nefarious object has been moved. Put another way, themethods and apparatus described in this section may be used to“authenticate” (trace or identify) known counterfeit objects.

The object 1904, once inducted as above, can then be moved, shipped,warehoused, bought, sold, etc. (block 1906) regardless of the legitimacyof its production or use at either the points of induction orauthentication. Dashed lines 1933A-1933B are intended to indicate that afield induction process may take place at any time or location duringthe life of the object. Whenever and wherever it occurs, the object 1950is thereafter traceable by using the stored authentication data.

In some embodiments, the datastore 1932 may be accessed or queried by anauthorized user 1940, which may be done remotely and programmatically.The present methods and apparatus may be used in at least the followingthree illustrative applications.

A. Controlled Induction, Stealth Authentication

In one illustration, police or other officials may have seized currencyor weapons during a criminal investigation. The officials induct theseized objects in case the objects are later stolen from their holdingfacility so as to make possible the tracking of the movements of thestolen objects and/or discovery of the thief. The induction may be donewith a field induction machine as described above. In another example,the seized evidence may be moved to a different location such as thepolice station, and inducted there. The field induction machine may belocated at such a location. An example would be the “Operation Fast andFurious”, except done correctly. Weapons are inducted (“fingerprinted”)at point of known provenance (manufacture, purchase, possession by agovernment agency) but there is risk of criminal use of a gun later. Anyauthentication will, however, be under far less controlledcircumstances, including possible in-field clandestine authentication.

FIG. 20 illustrates one example of this type of situation. A controlledinduction site 2200 may be a location of manufacture or of knownprovenance, 2202. An induction machine 2204 may be similar to the fieldinduction machine 1920 in FIG. 19, but here the induction machine 2204is located within the controlled induction environment. The dashed linesare to indicate, 2206A, the case in which the induction machine isintegrated into a manufacturing apparatus for inducting objectsimmediately after or even during manufacture, thus ensuring that onlygenuine objects are inducted. In another case, 2206B, the inductionmachine may be arranged for induction after manufacturer is completedbut before the object 2208 leaves the controlled induction environment2200. A fingerprint system 2210 may comprise a secure database server2211, coupled to a datastore 2212. The system 2210 may be locatedremotely, “in the cloud”, or at a physical location. A processor 2214may be configured to process image data received from the controlledinduction site 2200 via path 2216 and store extracted feature vectors inthe datastore 2212.

After the inducted object 2208 leaves the controlled environment 2200 itmay be subjected to various changes, intentionally, unintentionally, orthrough normal “wear and tear”. The use, handling, storage, transfer,etc. of the object are unknown, indicated by dashed line 2210. Thus anobject 2230 may or may not be the same object as 2208; but whether ornot it is may be determined by authentication.

Authentication may be implemented by a field authentication machine2232. A field authentication machine may comprise a scanner and otherelements similar to a field induction machine. Preferably, theauthentication machine 2232 is network connectable for transfer ofcaptured image data. The primary function is to capture image data,process it (locally or remotely), and compare the processed data,preferably comprising feature vectors, to data previously stored in adatastore. This may be done using a secure database server such as 2211.The data may be accessible to an authorized person or entity 2240. Forexample, a remote process (not shown) may query the database server foran authentication operation at another location. In some embodiments,the authentication machine 2232 may access the stored data and conductthe comparison. In another embodiment, the authentication machine 2232may submit the authentication data, for example, raw image data, to thesecure database server, for processing and then comparison to storeddata. Thus, the server 2211 may return an authentication result viacommunication path 2250 to the machine 2232. The processor 2214 and orthe database server 2211 may provide notifications if so configured,block 2252. Authentication will in some cases be under far lesscontrolled circumstances (than, say 2200) including possible in-fieldclandestine authentication.

B. Stealth Induction, Controlled Authentication

Another situation is illustrated by the currency example where acovertly captured digital fingerprint (a set of feature data or featurevectors) has been obtained at the point of creation but authenticationtakes place at a bank, customs office, or with the FBI.

C. Stealth Induction, Stealth Authentication.

In this example, both the induction and the authentication are doneclandestinely. In one example, known terrorists are seen moving weapons.Fingerprintable information is captured covertly without the terroristsknowing by some form of robotics, satellite, local agent, or other meansof clandestine collection. The fingerprint information may be collectedby scanning an object or document without removing it from itsthen-current location. Portable equipment may be used, including asmartphone or the like with a camera. Subsequently, another group isobserved moving arms. The same process occurs, but now the objects maybe authenticated, alerting security forces to the presence of the sameobjects in two known, illicit, locations as well as enabling the tracingof shipment paths and other associated factors. The process used may besimilar to the illustration of FIG. 19.

FIG. 21 is a simplified flow diagram of one example of a process forcreating a “digital fingerprint” that includes feature vectors based ona scanned image of an object. The process begins with initialization atblock 2120. This may comprise initializing a datastore, calibrating animage capture system, or other preliminary operations. An object or itemis scanned, block 2122, forming digital image data. From the image data,an authentication image is created, which may comprise all or a selectedsubset of the scan data. Next, a digital fingerprint may be initialized,for example in a memory or datastore.

To begin forming a digital fingerprint of the scanned object, at leastone authentication region is selected, block 2130, in the authenticationimage data. Each authentication region may be selected according to apredetermined template based on the class of objects. Locations of theauthentication regions may be stored in the fingerprint, block 2132.

At block 2134, the process continues by selecting locations of interestwithin each authentication region. To select locations in an image toextract fingerprint vectors, a software process automatically selects alarge number—preferably hundreds or even thousands per square mm—ofpreferred locations of interest for purposes of digital fingerprinting.A location may be of interest because of a relatively high level ofcontent. That “content” in a preferred embodiment may comprise agradient or vector, including a change in value and a direction.Selected areas of interest are added to the fingerprint, block 2136. Inone example, such areas may be identified by a location or centroid anda radius thus defining a circular region. Circular regions are preferredfor some applications because they are not affected by rotation of theimage.

Next, block 2138, the process calls for extracting features from eacharea of interest and forming feature vectors to describe those featuresin a compact form that facilitates later analysis, for example,calculation of vector distances as a metric of similarity in comparingfingerprints for authentication. Various techniques are known forextracting such features. The resulting feature vectors are added to thefingerprint, block 2140. At block 2142, additional information may beadded to the fingerprint, for example identifying other fingerprintsassociated with the same object. In some embodiments, a relationship,such as relative location of the other fingerprints to the currentfingerprint may be used. For example, in some objects, multiple regionsmay be authentic individually, but a change in their relative locationmay indicate that the object is not authentic. An example involving aweapon was given above.

The feature extraction may be repeated, block 2150, using an adjustedarea size or scale (magnification). Feature vectors created at theadjusted size may be added to the fingerprint, block 2152. Additionalfeatures may be extracted at additional magnification values, until anadequate number are provided, decision 2154. This additional data may beadded to the fingerprint, block 2156. This data may be helpful infinding a matching fingerprint in instances where the authenticationimage magnification is not the same as the image at the time ofinduction of the object. Finally, and optionally, the scanned imageitself (2122) may be added to the database, block 2158. This process tobuild a digital fingerprint ends at 2160.

Examples of Apparatus Embodiments

In some embodiments, an apparatus to implement features disclosed abovemay include: a capture device arranged to generate image data responsiveto a region of a physical object; a processor having electronic accessto the image data; the processor arranged to execute stored instructionsfor processing the image data, the stored instructions arranged to causethe processor to—

access a predetermined fingerprint template associated with a class ofobjects of which the physical object is or may be a member; wherein thefingerprint template specifies an authentication region, at least onelocation of interest within the authentication region, a method forextracting a feature vector characterizing the location of interest, anda method for comparing the extracted feature vector to a referencefeature vector;

based on the fingerprint template, identify a subset of the image datacorresponding to the authentication region; based on the fingerprinttemplate, extract a feature vector from the subset of the image data;and store extracted feature vector data defining the extracted featurevector in a database.

In some embodiments, an apparatus as described above may be modifiedwherein the capture device comprises at least one of a scanner, acamera, a specially-adapted sensor array such as CCD array, amicroscope, a smartphone camera, a video camera, an x-ray machine, asonar, an ultrasound machine, a microphone, or any other deviceresponsive to electromagnetic radiation or mechanical wave.

In some embodiments, an apparatus as described above may be modifiedwherein the stored instructions further control the processor to comparethe extracted feature vector data to reference feature vector data,without regard to the image data generated by the capture device, toestimate whether or not the physical object matches the referencefeature vector data.

In some embodiments, an apparatus as described above may be modifiedwherein the estimation includes quantifying the comparison and thencomparing the quantified comparison result to a predetermined tolerance.

In some embodiments, an apparatus as described above may be modifiedwherein the extracted feature vector data defines a contiguous geometricshape and a location of the geometric shape in the authenticationregion.

In some embodiments, an induction apparatus to implement featuresdisclosed above may include:

a scanner arranged to acquire digital image data representing an imageof at least a portion of a physical object positioned within anoperative distance from the scanner;

a digital processor coupled to the scanner to receive the digital imagedata;

a memory accessible to the digital processor and storing a fingerprinttemplate comprising data usable by the processor that specifies a classof objects and at least one authentication region for the class ofobjects;

the digital processor configured to execute instructions to—

locate an authentication region in the acquired image data based on thestored fingerprint template;

select a portion of the digital image data corresponding to the locatedauthentication region;

process the selected portion of the digital image data to identify alocation of interest within the located authentication region;

form a feature vector characterizing the location of interest;

store the feature vector in a database as part of a digital fingerprintof a scanned object.

In some embodiments, the induction apparatus as described above may bemodified wherein the authentication region comprises a hierarchy of oneor more authentication sub-regions, each sub-region containing at leastone location of interest.

In some embodiments, the induction apparatus as described above mayfurther include: a network interface coupled to the digital processor toenable data communications over a network; the network interfaceenabling the processor to receive and store a fingerprint template foruse by the apparatus in creating a digital fingerprint for a given classof objects.

In some embodiments, the induction apparatus as described above mayinclude:

a network interface coupled to the digital processor to enable datacommunications over a network; the network interface enabling theprocessor to transmit the feature vectors for storing the digitalfingerprint of a scanned object in a remote database system coupled tothe network.

In some embodiments, the induction apparatus as described above may bemodified wherein the scanner, the processor, and the memory are disposedin a portable system for data capture in the field.

In some embodiments, the induction apparatus as described above may bemodified wherein the scanner and the processor together implement imagemagnification in a range of approximately 10× to 40× the size of thephysical object.

In some embodiments, the induction apparatus as described above may bemodified wherein the scanner and the processor together implement imagemagnification that is selectable based on the fingerprint template.

It will be obvious to those having skill in the art that many changesmay be made to the details of the above-described embodiments withoutdeparting from the underlying principles of the disclosure. The scope ofthe present disclosure should, therefore, be determined only by thefollowing claims.

1. A computer-implemented method comprising: acquiring digital imagedata of at least a portion of a physical object; selecting at least oneauthentication region of the image data; analyzing the authenticationregion data to form a digital fingerprint of the object, wherein thedigital fingerprint comprises— for each authentication region, dataidentifying the authentication region, the data including a location ofthe authentication region in the image data; data identifying at leastone location of interest in the authentication region; for each locationof interest, data identifying at least one feature vector thatcharacterizes the location of interest; and storing the digitalfingerprint of the object in a database.
 2. The method of claim 1wherein the authentication region data comprises a hierarchy of one ormore authentication sub-regions, each sub-region containing at least onelocation of interest.
 3. The method of claim 1 wherein at least oneauthentication region is allocated a utility weight or ranking.
 4. Themethod of claim 1 wherein, for each location of interest, at least onecorresponding feature vector identifies coordinates of the location ofinterest and a digital characterization of the location of interest. 5.The method of claim 1 including automatically selecting theauthentication region and corresponding locations of interest inaccordance with a predetermined fingerprint template.
 6. The method ofclaim 5 wherein the fingerprint template comprises a stored datastructure that specifies a class of objects, at least one authenticationregion for the class of objects, and at least one location of interestwithin the authentication region for generating a digital fingerprint ofthe object.
 7. The method of claim 5 wherein the fingerprint templatedefines a method of extracting information from locations of interest,and a method for comparing the extracted information with potentiallymatching information.
 8. The method of claim 1 including storing thedigital fingerprint without storing the acquired digital image data inthe database.
 9. The method of claim 1 wherein extracting a featurevector comprises utilizing an algorithm among a class of algorithms nowexisting or later invented that is characterized by one or more of edgedetection, corner detection, blob detection, wavelet features, Gabor,gradient and steerable output filter histograms, scale-invariant featuretransformation, active contours, shape contexts, and parameterizedshapes.
 10. The method of claim 1 and further comprising: creating adigital fingerprint of a target object; comparing the digitalfingerprint of the target object to the stored digital fingerprint togenerate a result; and taking a predetermined action based on theresult.
 11. The method of claim 10 and wherein the comparing stepcomprises utilizing an algorithm of a set of algorithms characterized bynearest neighbor, hashing, indexing feature sets with visualvocabularies, support vector machines, multilayer preceptor, and randomforests and ferns.
 12. The method of claim 1 wherein the physical objectcomprises any one of government-issued documents, legal and financialdocuments, mail pieces, parcels, art, photographs, coins, currency,precious metals, gems, jewelry, apparel, mechanical parts, consumergoods, electronics, apparel, toys, integrated circuits, weapons,pharmaceuticals, drugs, chemicals, alcohol, tobacco, and food andbeverages.
 13. A computer-implemented method comprising: acquiringdigital fingerprints of a plurality of physical objects of a givenclass; and collecting the digital fingerprints to form amachine-readable database for use in analyzing objects of the sameclass.
 14. The method of claim 13 wherein the digital fingerprintscomprise digital fingerprints of a plurality of known or suspectedcounterfeit objects of the given class; and collecting the digitalfingerprints to form a machine-readable counterfeit object database foruse in detecting counterfeit or suspected counterfeit objects of thesame class.
 15. The method of claim 14 wherein each acquired digitalfingerprint comprises: data identifying at least one authenticationregion, the data including a location of the authentication region inthe image of the corresponding known or suspected counterfeit object;for each authentication region, data identifying at least one locationof interest in the authentication region; and for each location ofinterest, data identifying at least one feature vector thatcharacterizes the location of interest on the corresponding object. 16.The method of claim 15 wherein the authentication region comprises ahierarchy of one or more authentication sub-regions, each sub-regioncontaining at least one location of interest.
 17. The method of claim 14and further comprising: acquiring a digital fingerprint of a targetphysical object; querying the counterfeit object database based on thedigital fingerprint of the target physical object; and based on aresponse to the query, estimating a likelihood that the target physicalobject is a counterfeit.
 18. The method of claim 14 and furthercomprising: analyzing the digital fingerprints stored in the counterfeitobject database; and extracting patterns from the digital fingerprintsfor use in creating a database of known or suspected counterfeitcharacteristics for objects of the same class.
 19. The method of claim13 wherein the digital fingerprints comprise digital fingerprints of aplurality of objects of a given class, which objects conform to apredetermined object specification; and collecting the digitalfingerprints to form a machine-readable conforming object database foruse in comparing potentially conforming objects of the same class. 20.The method of claim 19 wherein each acquired digital fingerprintcomprises: data identifying at least one authentication region, the dataincluding a location of the authentication region in the image of thecorresponding conforming object; for each authentication region, dataidentifying at least one location of interest in the authenticationregion; and for each location of interest, data identifying at least onefeature vector that characterizes the location of interest on thecorresponding object.
 21. The method of claim 20 wherein theauthentication region comprises a hierarchy of one or moreauthentication sub-regions, each sub-region containing at least onelocation of interest.
 22. The method of claim 20 wherein at least one ofthe locations of interest is subject to a predetermined manufacturingspecification, and further comprising comparing the digital fingerprintto the predetermined manufacturing specification to determine the levelof compliance of the physical object to the predetermined manufacturingspecification.
 23. The method of claim 19 and further comprising:acquiring a digital fingerprint of a target physical object; queryingthe conforming object database based on the digital fingerprint of thetarget physical object; and based on a response to the query, estimatinga level of conformity of the target physical object.
 24. The method ofclaim 19 and further comprising: analyzing the digital fingerprintsstored in the conforming object database; and extracting patterns fromthe digital fingerprints to create a database of conformingcharacteristics for objects of the same class.
 25. The method of claim13 wherein the digital fingerprints comprise digital fingerprints of aplurality of objects of a given class, which objects are non-conformingto a predetermined object specification; and collecting the digitalfingerprints to form a machine-readable non-conforming object databasefor use in comparing potentially non-conforming objects of the sameclass.
 26. The method of claim 25 wherein each acquired digitalfingerprint comprises: data identifying at least one authenticationregion, the data including a location of the authentication region inthe image of the corresponding non-conforming object; for eachauthentication region, data identifying at least one location ofinterest in the authentication region; and for each location ofinterest, data identifying at least one feature vector thatcharacterizes the location of interest on the corresponding object. 27.The method of claim 26 wherein the authentication region comprises ahierarchy of one or more authentication sub-regions, each sub-regioncontaining at least one location of interest.
 28. The method of claim 25and further comprising: acquiring a digital fingerprint of a targetphysical object; querying the non-conforming object database based onthe digital fingerprint of the target physical object; and based on aresponse to the query, estimating a level of non-conformity of thetarget physical object.
 29. The method of claim 25 and furthercomprising: analyzing the digital fingerprints stored in thenon-conforming object database; and extracting patterns from the digitalfingerprints to create a database of non-conforming characteristics forobjects of the same class.